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(54) Image processing apparatus, image processing method, and information providing medium 

(57) An information processing apparatus, an infor- 
mation processing method, and an information provid- 
ing medium that allow a user to check the still pictures 
of a moving picture to be searched at desired points of 
time and desired time intervals and in a desired quan- 
tity. The user operates a mouse or a keyboard to enter a 
time and a time interval at which a picture is displayed. 
On the basis of these time and time interval, the CPU 
generates picture thumbnails of the moving picture. 
Then, the CPU simultaneously displays the generated 
picture thumbnails as arranged in a time sequential 
manner 
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Description 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001 ] The present invention relates generally to an 
image processing apparatus, an image processing 
method, and an information providing medium and. 
more particularly, to an image processing apparatus, an 
image processing method, and an information providing 
medium that search moving pictures and process the 
obtained moving pictures. 

Description of Related Art 

[0002] With a video tape recorder for example, pop- 
ularly known methods for searching scenes to be 
viewed or edited from a moving picture that takes long 
for reproduction include still, slow, queue, review, and 
other variable-speed reproduction techniques. Movie 
advertisements for example frequently use a technique 
of cutting out very short portions of a moving picture 
and connecting the cutouts to summarize the content of 
the original moving picture. 

[0003] For another searching method, video-on- 
demand uses a method of using a plurality of channels, 
which are switched between for reproducing and dis- 
playing a moving picture from a desired position at con- 
stant intervals. Further, for a method of searching a 
moving picture stored in a computer, a technique is 
known in which a plurality of still pictures included in a 
moving picture are simultaneously displayed on a mon- 
itor screen at predetermined constant time intervals. 
[0004] When the variable-speed display is used, 
only a moving picture at one time point can be displayed 
at a time in a picture search operation, thereby requiring 
a certain time for picture search. A moving picture in 
which the content of. a long moving picture requires a 
certain time for checking the content of the summary 
picture. In addition, a picture desired by a user is not 
always included in that summary picture. 
[0005] If the method of reproducing and displaying 
a moving picture from a given position at certain inter- 
vals is used, checking the content of the reproduced 
picture takes time and a desired picture is not always 
included in a selected channel. The method of display- 
ing a still picture included in a moving picture at certain 
time intervals requires to display many pictures; in this 
case, a desired picture is not always included in that still 
picture. 

SUMMARY OF THE INVENTION 

[0006] Thus, the known methods do not allow a 
user to check, at desired time intervals and in a desired 
quantity, the still pictures at given point of time, included 
in a moving picture to be searched, thereby makes it 



significantly difficult for the user to quickly search and 
use the content of moving pictures. 
[0007] It is therefore an object of the present inven- 
tion to provide an image processing apparatus, an 
5 image processing method, and an information providing 
medium that allow a user to check, at desired time inter- 
vals and in a desired quantity, still pictures at desired 
points of time, included in a moving picture to be 
searched. 

10 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0008] These and other objects of the invention will 
be seen by reference to the description, taken in con- 
15 nection with the accompanying drawing, in which: 

FIG. 1 is a block diagram illustrating a configuration 
of a personal computer practiced as a preferred 
embodiment of the invention; 
20 FIG. 2 is a diagram illustrating screens for search- 
ing moving pictures; 

FIG. 3 is a diagram illustrating images in more 
detail than those of FIG. 2; 
FIG. 4 is a diagram illustrating other screens for 
25 searching moving pictures; 

FIG. 5 is a diagram illustrating screens for display- 
ing pictures by inserting picture thumbnails of still 
smaller time intervals; 

FIG. 6 is a diagram illustrating the display of FIG. 5 
30 in more detail; 

FIG. 7 is a diagram illustrating an example of 
screens obtained by repeating picture thumbnails 
of shorter time intervals; 

FIG. 8 is a diagram illustrating screens obtained by 
35 deleting unnecessary image thumbnails; 

FIG. 9 is a diagram illustrating the images of FIG. 8 
in more detail; 

FIG. 10 is a diagram illustrating an example of 
reproducing a moving picture by specifying a 
40 searched picture thumbnail; 

FIG. 11 is a flowchart for describing the display 
processing; 

FIG. 12 is a flowchart for describing zoom-in 
processing for inserting picture thumbnails with pic- 
45 ture time interval narrowed; and 

FIG. 13 is a flowchart for describing zoom-out 
processing for expanding picture time intervals and 
deleting and inserting predetermined picture 
thumbnails. 

so 

DETAILED DESCRIPTION OF PREFERRED EMBOD- 
IMENTS 

[0009] This invention will be described in further 
55 detail by way of example with reference to the accompa- 
nying drawings. 

[001 0] In order to clarify the correlation between the 
means of the invention described in the claims 
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appended hereto and the components of the embodi- 
ment of the invention, each of the means is followed by 
parentheses in which an example of the corresponding 
component of the embodiment is enclosed. It should be 
noted however that each of the above means are not 
restricted to those described below. 
[0011] Namely, an image processing apparatus as 
cited in claim 1 appended hereto comprises an input 
means (for example, a mouse 17 shown in FIG. 1) for 
inputting the start and end points of an image to be dis- 
played, a thumbnail generating means (for example, a 
CPU 11 shown in FIG. 1) for generating a still picture 
(for example, a thumbnail) of a moving picture between 
the start and end points inputted by the input means at 
predetermined time intervals, and a display control 
means (for example, the CPU 11 shown in FIG. 1) for 
simultaneously displaying the predetermined number of 
thumbnails sequentially in time. 

[0012] Now, referring to FIG. 1, the CPU (Central 
Processing Unit) 11 executes an image processing 
application program and an OS (Operating System). A 
ROM (Read Only Memory) 12 stores basically fixed 
data among programs and computation parameters for 
use by the CPU 1 1 . A RAM (Random Access Memory) 
1 3 stores programs to be used by the CPU 1 1 for execu- 
tion and parameters which change from time to time in 
the execution. These components are interconnected 
by a bus 14. 

[001 3] A keyboard 1 6 is operated by a user to input 
a variety of commands into the CPU 1 1 . A mouse 1 7 is 
operated by the user to specify or select a point on the 
screen of a CRT (Cathode Ray Tube) 18. The CRT 18 
displays pictures and a variety of information in text or 
image. A HDD (Hard Disk Drive) 1 9 and an FDD (Floppy 
Disk Drive) 20 drive a hard disc and a floppy disc 
respectively on which programs to be executed by the 
CPU. 11 and information are stored or reproduced. An 
optical disc driver 21 is loaded with an optical disc on 
which moving pictures are stored. These components 
are connected to an interface 15 which is connected to 
the CPU 1 1 through the bus 14. 
[0014] The following describes the screens to be 
displayed when a moving picture is searched by use of 
the personal computer practiced as one preferred 
embodiment with reference to FIG. 2. FIG. 2 shows 15 
picture thumbnails created from still pictures at intervals 
of one minute of a moving picture having a duration of 
15 minutes, these picture thumbnails being simultane- 
ously displayed on the CRT 18. Namely, the picture 
thumbnail located at the upper leftmost of the diagram 
indicates the first picture of the moving picture (namely, 
the picture obtained at location which is zero minute 
zero second after the start picture (origin) of the moving 
picture). The second picture thumbnail, next to the first 
picture on top row, indicates a picture obtained one 
minute zero second after the origin. Thus, each thumb- 
nail on top row indicates a picture obtained one minute 
zero second after the thumbnail immediately on the left 



side. Therefore, the five picture thumbnails on top row 
indicate the pictures obtained four minutes zero second 
in total after the first or origin picture of the moving pic- 
ture. 

5 [001 5] The picture thumbnail located at the leftmost 
of middle row indicates a picture obtained one minute 
zero second after the picture thumbnail located at the 
rightmost of top row. Namely, the middle left thumbnail 
indicates a picture obtained five minutes zero second 

io after the origin. Each picture thumbnail on middle row 
also indicates a picture obtained one minute zero sec- 
ond after the picture thumbnail immediately on the left 
side. Therefore, the five picture thumbnails on middle 
row range from the picture obtained five minutes zero 

15 second after the origin to the picture nine minutes zero 
second after the origin. 

[001 6] The picture thumbnail located at the leftmost 
of bottom row indicates a picture obtained one minute 
zero second after the picture thumbnail located at the 

20 rightmost of middle row, namely the picture obtained 1 0 
minutes zero second after the origin. Each picture 
thumbnail on bottom row also indicates a picture 
obtained one minute zero second after the immediately 
left-side picture. Therefore, the five picture thumbnails 

25 on bottom row range from the picture obtained 10 min- 
utes zero second after the origin to the picture obtained 
14 minutes zero second after the origin. 
[0017] Thus, the CRT 18 simultaneously displays 
all the picture thumbnails obtained at intervals of one 

30 minute of the moving picture. Obviously, the number of 
picture thumbnails need not be limited to 15. As long as 
the content is identifiable, the CRT 18 can display any 
number of picture thumbnails in one screen or the CRT 
1 8 can change the picture thumbnails to be displayed by 

35 scrolling. In addition, in the above-mentioned example, 
the pictures obtained every one minute are used. The 
time interval, however, may be altered to 10 minutes or 
one second for example as required. 
[0018] FIG. 3 shows picture thumbnails in more 

40 detail than those shown in FIG. 2. The CRT 1 8 displays 
various buttons necessary for a picture thumbnail oper- 
ation and the predetermined number of picture thumb- 
nails sequentially arranged in time. 
[0019] The following describes another screen 

45 example for moving picture search by use of the per- 
sonal computer associated with the present invention, 
with reference to FIG. 4. FIG. 4 shows the CRT 18 
simultaneously displaying the picture thumbnails of six 
moving pictures. On top row, there are five picture 

so thumbnails in total at time intervals of five minutes of the 
world news moving picture. On the second row from top, 
there are five picture thumbnails in total at intervals of 
10 minutes of the economic news moving picture. On 
the third row, there are five pictures in total at internals 

55 of one minute of the political mews moving picture. 
[0020] On the fourth row, there are five picture 
thumbnails- in total at intervals of three minutes of the 
animation moving picture. On the fifth row, there are five 
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picture thumbnails at intervals of 20 minutes of the 
movie moving picture. On the sixth or bottom row, there 
are five picture thumbnails at intervals of 15 minutes of 
the sports moving picture. 

[0021] Thus, the CRT 18 simultaneously displays 
the picture thumbnails at predetermined intervals of 
each of the plural moving pictures. 
[0022] FIG. 5 shows a state in which picture thumb- 
nails at smaller time intervals than those shown in FIG. 
2 are inserted. Namely, between the picture thumbnail 
indicative of the picture obtained eight minutes zero 
second after the origin and the picture thumbnail indica- 
tive of the picture obtained nine minutes zero second 
after the origin, 1 1 picture thumbnails at time intervals of 
five seconds are displayed. 

[0023] Thus, picture thumbnails of shorter time 
intervals may be inserted, at desired time, below the top 
row of predetermined picture thumbnails. 
[0024] FIG. 6 illustrates an image in more detail 
than that shown in FIG. 5. In the image of FIG. 6, prede- 
termined picture thumbnails are inserted in the display 
shown in FIG. 3. 

[0025] FIG. 7 shows an example in which the inser- 
tion of picture thumbnails of shorter time intervals 
described with reference to FIG. 5 is repeated. As 
shown, the display of picture thumbnails obtained at 
intervals of 10 minutes from a moving picture having a 
duration of 100 minutes is inserted with seven picture 
thumbnails at intervals of one minute with the origin 40 
minutes zero second passed from the reproduction of 
the moving picture (namely, a picture 41 minutes zero 
second after the origin, a picture 42 minutes zero sec- 
ond after, a picture 43 minutes zero second after, a pic- 
ture 44 minutes zero second after, a picture 45 minutes 
zero second after, a picture 46 minutes zero second 
after, and a picture 47 minutes zero second after). 
[0026] Then, a picture thumbnail indicative of a pic- 
ture 43 minutes 30 seconds passed from the origin fol- 
lows the image 43 minutes zero second passed from 
the reproduction of the moving picture. 
[0027] Further, with the image 44 minutes zero sec- 
ond passed from the reproduction of the moving picture 
being used as the origin, eight picture thumbnails at 
intervals of 5 seconds are inserted (namely, a picture 44 
minutes 5 second after the origin, a picture 44 minutes 
1 0 seconds after, a picture 44 minutes 1 5 seconds after, 
a picture 44 minutes 20 seconds after, a picture 44 min- 
utes 25 seconds after, a picture 44 minutes 30 seconds 
after, a picture 44 minutes 35 seconds after, and a pic- 
ture 44 minutes 40 seconds after). 
[0028] Lastly, with a picture 44 minutes 20 seconds 
passed from the reproduction of the moving picture 
being used as the origin, four picture thumbnails at 
intervals of one second are inserted (namely, a picture 
44 minutes 21 seconds after the origin, a picture 44 min- 
utes 22 seconds after, a picture 44 minutes 23 seconds 
after, and a picture 44 minutes 24 seconds after). 
[0029] Thus, narrowing the picture display time 



intervals allows the user operating the personal compu- 
ter to quickly search the middle row of a long moving 
picture for desired pictures. 

[0030] The picture thumbnails not required for pic- 

5 ture search can be deleted as shown in FIG. 8. Namely, 
as shown in FIG. 8, the picture thumbnails starting with 
the picture thumbnail one minute zero second passed 
from the origin and ending with the picture thumbnail 4 
minutes zero second passed from the origin are deleted 

io from the state shown in FIG. 2. 

[0031] FIG. 9 shows an image in more detail than 
that of FIG. 8. Compared with the example shown in 
FIG. 3, the example of FIG. 8 is smaller in the number of 
displayed picture thumbnails. 

is [0032] FIG. 10 shows a screen example in which a 
searched picture thumbnail is specified to reproduce the 
moving picture. As shown, a moving picture display win- 
dow is opened on top of picture thumbnails. In the upper 
portion of the window, the moving picture is displayed 

20 that starts from the picture indicated by the specified 
picture thumbnail. Two picture thumbnails may be spec- 
ified to reproduce the moving picture in between. 
[0033] The following describes the above-men- 
tioned display processing with reference to the flow- 

25 chart shown in FIG. 1 1 . In step S1 1 , the user operates 
the keyboard 16 or the mouse 17 to select a desired 
moving picture from among those stored on the optical 
disc loaded in the optical disc driver 21 . In step S12, the 
user operates the keyboard 1 6 or the mouse 1 7 to input 

30 the time intervals for the pictures to be displayed. 

[0034] In step S13, the CPU 1 1 creates the picture 
thumbnails of the selected moving picture at the time 
interval inputted in step Si 2. In step S14, the CPU 1 1 
displays the picture thumbnails created in step S1 3 onto 

35 the CRT 18 in a time-sequentially arranged manner, 
upon which the processing comes to an end. 
[0035] Thus, the CRT 18 of the personal computer 
simultaneously displays the picture thumbnails of the 
moving picture at the predetermined intervals as 

40 arranged in a time sequential manner. 

[0036] The following describes zoom-in processing 
for displaying picture thumbnails with the picture time 
interval narrowed, with reference to the flowchart shown 
in FIG. 12. In step S21, the user operates the keyboard 

45 16 or the mouse 17 to select a picture thumbnail from 
which zoom-in processing will start, tn step S22, the 
user operates the keyboard 16 or the mouse 17 to 
select a picture thumbnail at which the zoom-in 
processing will end. In step S23, the user operates the 

so keyboard 1 6 or the mouse 1 7 to set zoom-in time inter- 
vals. 

[0037] In step S24, the CPU 11 creates picture 
thumbnails of the picture between the picture thumb- 
nails specified in steps S21 and S22, at the time inter- 
55 vals inputted in step S23. In step S25, the CPU 11 
inserts the picture thumbnails created in step S24 
between the original thumbnails in a time sequential 
manner and displays this state onto the CRT 1 8. At this 
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moment, the picture thumbnails located backward in 
time are sequentially shifted backward. 
[0038] Thus, the picture thumbnails with time inter- 
val narrowed inserted between desired pictures can be 
displayed. 

[0039] The following describes zoom-out process- 
ing for increasing a picture time interval and deleting 
and inserting desired thumbnails, with reference to the 
flowchart shown in FIG. 13. In step S31, the user oper- 
ates the keyboard 1 6 or the mouse 1 7 to select a picture 
thumbnail from which zoom-out processing will start. In 
step S32, the user operates the keyboard 16 or the 
mouse 17 to select a thumbnail at which the zoom-out 
processing will end. In step S33, the user operates the 
keyboard 1 6 or the mouse 1 7 to set zoom-out time inter- 
vals. 

[0040] In step S34, the CPU 1 1 deletes unneces- 
sary picture thumbnails of the picture between the pic- 
ture thumbnails specified in steps S31 and S32, at the 
time intervals inputted in step S33. In step S35, the CPU 
1 1 creates the picture thumbnails of the picture between 
the picture thumbnails specified in step S31 and S32, at 
the time intervals inputted in step S33. In step S36, the 
CPU 1 1 inserts the picture thumbnails created in step 
S35 between the original picture thumbnails in a time 
sequential manner and displays this state on the CRT 
18. At this moment, the picture thumbnails arranged 
backward in time are sequentially shifted forward. 
[0041] Thus, the picture thumbnails with time inter- 
val expanded between desired pictures can be dis- 
played. 

[0042] In the above-mentioned example, time inter- 
vals are inputted by the user. It will be apparent that pre- 
set time intervals may be used. In this case, the picture 
thumbnails may be displayed between the start point 
and end point specified by the user at the preset inter- 
vals. In this case, one of zoom-in mode or zoom-out 
mode is selected in advance. 

[0043] In the above-mentioned example, the size of 
picture thumbnails is preset and the maximum number 
of picture thumbnails to be displayed on the CRT 18 is 
constant. It will be apparent that the size may be varia- 
ble with the number of picture thumbnails to be dis- 
played (namely, the size is smaller as the number of 
picture thumbnails is large and vice versa). Also, the 
number of picture thumbnails to be displayed on the 
CRT 18 may be specified by the user and the size of the 
picture thumbnails may be varied accordingly. 
[0044] Thus, with the personal computer associ- 
ated with the invention, necessary picture thumbnails at 
a plurality of desired time points of a moving picture are 
simultaneously displayed at desired time intervals. This 
novel constitution allows the user to quickly search 
desired pictures. 

[0045] Further, the user may specify a plurality of 
selected picture thumbnails not only to perform moving 
picture reproduction but also open a moving picture 
editing window on top of the screen for moving picture 



cutout, copy, and paste operations. 
[0046] It should be noted that a medium for provid- 
ing the user with the computer programs for executing 
the above-mentioned processing includes not only 

s recording media such as magnetic disc, CD-ROM, and 
solid memory but also communication media such as 
network and communication satellite. 
[0047] As described and according to the image 
processing apparatus cited in claim 1 appended hereto, 

io the image processing method cited in claim 5 appended 
hereto, and the information providing medium cited in 
claim 9 appended hereto, the start and end points of a 
picture to be displayed are inputted by the user, the pic- 
ture thumbnails between the inputted start and end 

is points are generated at predetermined time intervals, 
and the predetermined number of thumbnails are simul- 
taneously displayed as arranged in a time sequential 
manner. This novel constitution allows the user to check 
the pictures of a moving picture to be searched at 

20 desired points of time and desired time intervals and in 
a desired quantity. 

[0048] While the preferred embodiments of the 
present invention have been described using specific 
terms, such description is for illustrative purposes only, 
25 and it is to be understood that changes and variations 
may be made without departing from the spirit or scope 
of the appended claims. 

Claims 

30 

1 . An image processing apparatus for displaying a still 
picture sequence constituting a series of moving 
pictures along an elapsed time of said still picture 
sequence, comprising: 

35 

a range specification means for specifying a 
desired range of said series of moving pictures; 
a still picture generating means for forming a 
still picture sequence at a predetermined 
40 elapsed time from among the moving pictures 

in the range specified by said range specifica- 
tion means; and 

a display means for displaying said still picture 
sequence generated by said still picture gener- 
45 ating means along said elapsed time. 

2. The image processing apparatus as claimed in 
claim 1 , further comprising: 

so a time interval setting means for setting said 

predetermined elapsed time. 

3. Trie image processing apparatus as claimed in 
claim 1, wherein said predetermined elapsed time 

55 is calculated from the number of still pictures that 
can be displayed at a predetermined display loca- 
tion. 
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4. The image processing apparatus as claimed in 
claim 1 , wherein, if said range is specified with said 
still picture sequence already displayed on said dis- 
play means, any preceding still picture is deleted for 
still picture generation. 5 

5. An image processing method for displaying a still 
picture sequence constituting a series of moving 
pictures along an elapsed time of said still picture 
sequence, comprising the steps of: io 

specifying a desired range of said series of 
moving pictures; 

forming a still picture sequence at a predeter- 
mined elapsed time from among the moving is 
pictures in the range specified in the range 
specification step; and 

displaying said still picture sequence gener- 
ated in the still picture generating step along 
said elapsed time. 20 



11. The information providing medium as claimed in 
claim 9, wherein said predetermined elapsed time 
is calculated from the number of still pictures that 
can be displayed at a predetermined display loca- 
tion. 

12. The information providing medium as claimed in 
claim 9, wherein, if said range is specified with said 
still picture sequence already displayed in said dis- 
play step, any preceding still picture is deleted for 
still picture generation. 



6. The image processing method as claimed in claim 
5, further comprising the step of: 

setting said predetermined elapsed time. 25 

7. The image processing method as claimed in claim 
5, wherein said predetermined elapsed time is cal- 
culated from the number of still pictures that can be 
displayed at a predetermined display location. 30 



8. The image processing method as claimed in claim 
5, wherein, if said range is specified with said still 
picture sequence already displayed in said display 
step, any preceding still picture is deleted for still 35 
picture generation. 



9. An information providing medium for providing a 
computer program for displaying a still picture 
sequence constituting a series of moving pictures <o 
along an elapsed time of said still picture sequence, 
said computer program comprising the steps of: 



specifying a desired range of said series of 
moving pictures; 45 
forming a still picture sequence at a predeter- 
mined elapsed time from among the moving 
pictures in the range specified in the range 
specification step; and 

displaying said stilt picture sequence gener- so 
ated in the still picture generating step along 
said elapsed time. 



10. The information providing medium as claimed in 
claim 9, wherein said computer program further ss 
comprising the step of: 

setting said predetermined elapsed time. 
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